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DETAILED ACTION 



1 . This Office Action is responsive to the following submissions: 

The substitute specification filed on October 6, 2000; 

Amendment A, filed on October 6, 2000; 

Amendment B, filed on October 17, 2002; and 

Amendment C, filed on November 29, 2002; 
All of which have been entered. Claims 1-51 and 53-61 are now pending. 



2. The following quotations of 37 CFR § 1 .75(a) is the basis of objection: 

(a) The specification must conclude with a claim particularly pointing but and distinctly claiming 
the subject matter which the applicant regards as his invention or discovery. 

3. Claims 51 and 58 are objected to under 37 CFR § 1 .75(a) as failing to particularly 
point out and distinctly claim the subject matter which the applicant regards as his 
invention or discovery. Claim 51 seems to recite an equation, but the equation itself is 
missing. The equation of claim 20 will be assumed for examination purposes. Claim 58 
depends from a cancelled claim (i.e., claim 52). Claim 58 will be assumed to depend 
from claim 37 for examination purposes. 



Claim Objections 
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Claim Rejections - 35 USC § 102 



4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 



(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



5. Claim 10 is rejected under 35 U.S.C. 102(e) as being anticipated by Jacquin et 
al. (US 5,764,803 A). 

Regarding claim 10, Jacquin discloses detecting a facial region ("face" at column 
2, line 15) within a video ("video" at column 1, line 8) comprising (Note: preamble given 
weight because it is referred to in the body, and breaths life into the claim): 

calculating a difference between a first and subsequent frame (figure 1, numeral 

20); 

determining plural candidate facial regions within the difference image 
("candidate ellipses" at figure 2); and 

fitting the candidate facial regions to the difference image to select one of the 
candidate facial regions ("chosen ellipse" at figure 2; "fitness metric" at column 7, line 
15), based on a combination of three factors, including a fit factor representing a fit of 



States. 
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the candidate ellipse to the difference image ("dborder" at column 7, line 19, equation 12), 
a location factor representative of the location of the candidate facial regions within the 
video (this limitation is equally anticipated by at least three elements in Jacquin: first, by 
"dmotion" at column 7, line 19, equation 12; second, by "separation measure D" at column 
9, line 10; and third, by "location" at column 8, line 17), and a size factor representative 
of the size of the candidate facial regions (this limitation is equally anticipated by at least 
two elements in Jacquin: first, "Pmotion" at column 7, line 19, equation 12, and second, 
"size and shape" at column 8, line 18). 

6. Claims 21 , 22, 23, 26-29, 31 , 33, 37, 53, 54 and 58-60 are rejected under 35 
U.S.C. 102(e) as being anticipated by Ryoo (US 5,990,957 A). 

Regarding claims 28, 37 and 59, Ryoo discloses a method for encoding video 
("video coding" at column 1 , line 7; preamble given weight because it is referred to in 
the claim body) comprising: 

detection a location of a facial regions of a fame of video (a frame is segmented 
into "video object planes", or "VOPs" at column 2, line 16, as depicted in figure 5B; once 
such VOP is a facial region; i.e., "a face portion" at column 12, line 23); 

calculating a sensitivity value for each of a plurality of locations within the frame 
(for each VOP, a sensitivity value is added or subtracted to the target bit allocation for a 
specific VOPs as described at column 1 1 , lines 5-35; that is, bit are "incremented for 
important VOPs" at column 1 1 , line 10; see "no_of_addJ)its" at column 1 1 , line 23; this 
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number is then subtracted from unimportant VOPs; once such important VOP is that of 
the facial region as described at column 12, line 23); and 

encoding the frame in a manner that provides substantially uniform apparent 
quality to perceiving detail at eccentric visual angles of the plurality of locations to the 
viewer when the viewer is observing the facial region (FIRST: "picture quality is 
enhanced ... since a uniform picture quality is generally maintained even when the 
compression rate is different between each portions of a picture in view of video 
characteristics" at column 12, lines 28-32; this is because the coding of all image areas 
in done in accordance with a "visual sensitivity classifier" at figure 2, numeral 23, which 
takes into account "human visual sensitivity" at column 2, line 21; SECOND: facial 
region is the subject of the video and what the viewer will focus on; so by encoding that 
region with finer quantization [i.e., less compression], the inherent property of the 
human visual system to perceive finer detail at the center of the field, and less detail in 
the periphery of vision is taken advantage of; therefore, even with coarser compression 
in the periphery [i.e., outside of the facial region], the apparent quality will be uniform). 

Regarding claims 27 and 58, the total number of bits per frame is kept constant 
("target bit rate of the entire frame is kept constant" at column 10, line 46; "entire bit 
amount can be kept constant" at column 12, line 18). 

Regarding claims 22, 23, 29, 53, 54, and 60, the encoding of the locations is 
based on a quantization value representative of a base quantization factor divided by 
sensitivity information (the target bit rate is divided by frame texture target bits, which is 
sensitivity information, at column 11, line 33). 
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Regarding claims 21, 22 and 31, the limitations recited therein are anticipated by 
Ryoo as described above. Further, and commensurate with the claim requirements, 
Ryoo adaptively quantizes and encodes "blocks" (i.e., "each macroblock" at column 5, 
line 27; "each block" at column 1 1 , line 44). 

Regarding claims 26 and 33, quantization values are adjusted according to the 
number of blocks remaining to be encoded ("target bits" at column 1 1 , line 42), number 
of bits still available ("buffer occupancy" at column 1 1 , line 44), and sensitivity and 
texture of the remaining blocks ("visual sensitivity ... variance for each block" at column 
11, line 43). 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 1, 4, 6 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Jacquin et al. (US 5,764,803 A) and McLaughlin (Randomized 
Hough Transform: Better Ellipse Detection). 

The Jacquin Reference 

Regarding claims 1 and 10, Jacquin discloses detecting a facial region ("face" at 
column 2, line 15) within a video ("video" at column 1 , line 8) comprising (Note: 
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preamble given weight because it is referred to in the body, and breaths life into the 
claim): 

calculating a difference between a first and subsequent frame (figure 1 , numeral 

20); 

determining plural candidate facial regions within the difference image 
("candidate ellipses" at figure 2); and 

fitting the candidate facial regions to the difference image to select one of the 
candidate facial regions ("chosen ellipse" at figure 2; "fitness metric" at column 7, line 
15), where the difference image used for the fitting is free from being transformed as a 
result of the determining step (once the candidate ellipses are determined, then Jacquin 
uses a "fitness metric" at column 7, line 15 and equation 12 to compute the fitness of 
each of the candidate ellipses with the difference image directly; for example, Jacquin 
states, "measure of the density of edge data on a candidate ellipse border (i.e., the 
percentage of the pixels on the border which have classified as edge pixels be being 
assigned the value b2)" at column 7, line 20; that is, the density of the "pixel" data of the 
original difference image that are on the border of the candidate ellipse is measured; 
thus, in the "fitting" step, Jacquin uses the original, untransformed difference image). 

Regarding claim 6, the fitting is based on a combination of three factors, 
including a fit factor representing a fit of the candidate ellipse to the difference image 
("dborder" at column 7, line 19, equation 12), a location factor representative of the 
location of the candidate facial regions within the video (this limitation is equally 
anticipated by at least three elements in Jacquin: first, by "d mo ti 0 n" at column 7, line 19, 



Application/Control Number: 09/661 ,633 Page 8 

Art Unit: 2621 

equation 12; second, by "separation measure D" at column 9, line 10; and third, by 
"location" at column 8, line 1 7), and a size factor representative of the size of the 
candidate facial regions (this limitation is equally anticipated by at least two elements in 
Jacquin: first, "P mo tion" at column 7, line 19, equation 12, and second, "size and shape" 
at column 8, line 18). 

Differences 

Regarding claim 1, Jacquin's does not teach determining the plural candidate 
facial regions within the difference image based on a transform of the difference 
image in a spatial domain to a parameter space. Note: Jacquin does not specifically 
disclose how the initial candidate ellipses are formed. All Jacquin states is that "ellipse 
finder 44 generates candidate ellipses based on the foreground motion-and-edge image 
data" at column 6, line 61 . 

Regarding claims 4 and 10, Jacquin does not teach a Hough transform. 

The McLaughlin Reference 

Regarding claim 1 , McLaughlin discloses a system in the field of image 
processing, and in the same problem solving area of finding ellipses in an image ("then 
the ellipse is judged to exist" at page 41 1 , right column), comprising determining ellipses 
within an image based on a transform of the image in a spatial domain to a parameter 
space ("p, q, r1 , r2, 0" and "5D parameter space" at page 41 1 , right column). 
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Regarding claims 4 and 10, McLaughlin's parameter space is Hough space 
("Hough Transform for ellipse detection" at page 412, left column, bottom paragraph). 

The Combination 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the Hough parameter space ellipse finding method taught by 
McLaughlin, as the method of generating the candidate ellipses required but unspecified 
by Jacquin (i.e., as described by Jacquin at column 6, line 61), because the McLaughlin 
has "demonstrated high accuracy" and it has "shown tolerance to certain types of noise 
and partial occlusion" (McLaughlin page 414, left column), thus ensuring that the 
potential face regions of Jacquin are not missed even when in noisy image or when 
partially occluded by other objects in the image (as faces sometimes are). 

9. Claims 2 and 3 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Jacquin et al. (US 5,764,803 A) and McLaughlin (Randomized 
Hough Transform: Better Ellipse Detection) as applied to claim 1 , and further in 
combination with Sexton (US 5,086,480 A). 

Regarding claims 2 and 3, Jacquin does not threshold the difference image by 
setting values of the difference image that are less than a threshold to a selected value. 

Sexton discloses a system that detects faces in an image sequence (e.g., figures 
2), comprising subtracting adjacent frames (figure 1 , numeral 2), where Sexton teaches 
the further step of thresholding the difference image (figure 1, numeral 3) by setting 
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values of the difference image that are less than a threshold to a selected value ("each 
pel below the threshold is set to minimum intensity (0)" at column 3, line 1 ). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to incorporate into the algorithm of Jacquin, following the image 
subtraction step at figure 1 , numeral 20, the additional step of thresholding the 
difference image as taught by Sexton, in order to remove "a large quantity of the 
random noise" (Sexton, column 3, line 3) and thus improve the accuracy of facial 
detection and reduce processing time by removing noisy regions that are irrelevant to 
the extraction of a facial region. 

1 0. Claim 1 3 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over the 
combination of Jacquin et al. (US 5,764,803 A) and Sexton (US 5,086,480 A). 

Regarding claim 13, Jacquin does not threshold the difference image by setting 
values of the difference image that are less than a threshold to a selected value. 

Sexton discloses a system that detects faces in an image sequence (e.g., figures 
2), comprising subtracting adjacent frames (figure 1, numeral 2), where Sexton teaches 
the further step of thresholding the difference image (figure 1, numeral 3) by setting 
values of the difference image that are less than a threshold to a selected value ("each 
pel below the threshold is set to minimum intensity (0)" at column 3, line 1 ). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to incorporate into the algorithm of Jacquin, following the image 
subtraction step at figure 1, numeral 20, the additional step of thresholding the 
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difference image as taught by Sexton, in order to remove "a large quantity of the 
random noise" (Sexton, column 3, line 3) and thus improve the accuracy of facial 
detection and reduce processing time by removing noisy regions that are irrelevant to 
the extraction of a facial region. 

1 1 . Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Jacquin et al. (US 5,764,803 A) and McLaughlin (Randomized Hough 
Transform: Better Ellipse Detection) as applied to claim 1, and further in combination 
with Eleftheriadis et al. (US 5,852,669 A). 

Regarding claim 7, Jacquin does not reduce the number of pixels in the frames 
prior to calculating the difference frame. 

Eleftheriadis discloses a system that detects faces in an image sequence (e.g., 
figures 3 and 5), comprising pre-processing the image frames by reducing the number 
of pixels in the frames (figure 5, numeral 94 includes decimator 122 at figure 6, which 
reduces the image sizes; e.g., "decimation factor c" at column 6, line 4). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to reduce the image sizes of the current and previous frames of Jacquin 
(i.e., as depicted in figure 1) as a pre-processing step as taught by Eleftheriadis, to 
"provide for a low computational complexity" and thus avoid "exhaustive searches" 
(Eleftheriadis, column 6, lines 49-52) by virtue of having to process fewer pixels, and 
thus less data which still retaining the important face like regions in the image. 
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12. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Jacquin et al. (US 5,764,803 A) and Eleftheriadis et al. (US 5,852,669 
A). 

Regarding claim 14, Jacquin does not reduce the number of pixels (i.e., scale) in 
the frames prior to calculating the difference frame. 

Eleftheriadis discloses a system that detects faces in an image sequence (e.g., 
figures 3 and 5), comprising pre-processing the image frames by reducing the number 
of pixels in the frames (figure 5, numeral 94 includes decimator 122 at figure 6, which 
reduces the image sizes; e.g., "decimation factor c" at column 6, line 4). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to reduce the image sizes of the current and previous frames of Jacquin 
(i.e., as depicted in figure 1) as a pre-processing step as taught by Eleftheriadis, to 
"provide for a low computational complexity" and thus avoid "exhaustive searches" 
(Eleftheriadis, column 6, lines 49-52) by virtue of having to process fewer pixels, and 
thus less data which still retaining the important face like regions in the image. 

13. Claims 39 and 44 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ryoo (US 5,990,957 A) as applied to claim 37 above, in combination with Jacquin 
et al. (US 5,764,803 A) as applied to claim 10 above. 

While Ryoo extracts a facial region for coding as described above, Ryoo does 
not teach the specific method of extracting the facial region as recited in claim 37. 

Jacquin teaches just such a method as described in the claim 10 rejection above. 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize, as the face regions extraction method required by Ryoo, the 
method taught by Jacquin, in order to provide a facial extraction method that is "well 
suited to scenes having static, albeit complex, backgrounds" (Jacquin, column 2, line 
30), which provides "improved tracking of the objects of interest" (Jacquin, column 3, 
line 7). 

14. Claim 40 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ryoo 
(US 5,990,957 A) and Jacquin et al. (US 5,764,803 A) as applied to claim 39, and 
further in combination with McLaughlin (Randomized Hough Transform: Better Ellipse 
Detection). 

In the Ryoo and Jacquin combination as applied to claim 39 above, while Jacquin 
determines a plurality of candidate regions, Jacquin does not teach a Hough transform. 

McLaughlin discloses a system in the field of image processing, and in the same 
problem solving area of finding ellipses in an image ("then the ellipse is judged to exist" 
at page 41 1 , right column), comprising determining ellipses within an image based on a 
transform of the image in a spatial domain to a parameter space ("p, q, r1, r2, 0" and 
"5D parameter space" at page 41 1 , right column), where the parameter space is Hough 
space ("Hough Transform for ellipse detection" at page 412, left column, bottom 
paragraph). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the Hough parameter space ellipse finding method taught by 
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McLaughlin, as the method of generating the candidate ellipses required but unspecified 
by Jacquin (i.e., as described by Jacquin at column 6, line 61), because the McLaughlin 
has "demonstrated high accuracy" and it has "shown tolerance to certain types of noise 
and partial occlusion" (McLaughlin page 414, left column), thus ensuring that the 
potential face regions of Jacquin are not missed even when in noisy image or when 
partially occluded by other objects in the image (as faces sometimes are). 

15. Claims 40 and 42 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ryoo (US 5,990,957 A) and Jacquin et al. (US 5,764,803 A) as applied to claim 39, 
and further in combination with Sexton (US 5,086,480 A). 

In the Ryoo and Jacquin combination as applied to claim 39 above, while Jacquin 
calculates a difference image, Jacquin does not threshold the difference image by 
setting values of the difference image that are less than a threshold to a selected value. 

Sexton discloses a system that detects faces in an image sequence (e.g., figures 
2), comprising subtracting adjacent frames (figure 1, numeral 2), where Sexton teaches 
the further step of thresholding the difference image (figure 1, numeral 3) by setting 
values of the difference image that are less than a threshold to a selected value ("each 
pel below the threshold is set to minimum intensity (0)" at column 3, line 1 ). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to incorporate into the algorithm of Jacquin, following the image 
subtraction step at figure 1 , numeral 20, the additional step of thresholding the 
difference image as taught by Sexton, in order to remove "a large quantity of the 
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random noise" (Sexton, column 3, line 3) and thus improve the accuracy of facial 
detection and reduce processing time by removing noisy regions that are irrelevant to 
the extraction of a facial region. 

16. Claim 45 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ryoo 
(US 5,990,957 A) and Jacquin et al. (US 5,764,803 A) as applied to claim 39, and 
further in combination with Eleftheriadis et al. (US 5,852,669 A). 

In the Ryoo and Jacquin combination as applied to claim 39 above, while Jacquin 
calculates a difference image, Jacquin does not reduce the number of pixels (i.e., scale) 
in the frames prior to calculating the difference frame. 

Eleftheriadis discloses a system that detects faces in an image sequence (e.g., 
figures 3 and 5), comprising pre-processing the image frames by reducing the number 
of pixels in the frames (figure 5, numeral 94 includes decimator 122 at figure 6, which 
reduces the image sizes; e.g., "decimation factor c" at column 6, line 4). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to reduce the image sizes of the current and previous frames of Jacquin 
(i.e., as depicted in figure 1 ) as a pre-processing step as taught by Eleftheriadis, to 
"provide for a low computational complexity" and thus avoid "exhaustive searches" 
(Eleftheriadis, column 6, lines 49-52) by virtue of having to process fewer pixels, and 
thus less data which still retaining the important face like regions in the image. 



A 
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Allowable Subject Matter 



1 7. Claims 1 5-20 are allowed. Independent claim 1 5 distinguishes over the prior art 
in that the prior art does not teach determining a spatial location of a facial region, and 
then calculating a sensitivity value for each of a plurality of spatial locations based on: 

the spatial location of the facial region in relation to the plurality of spatial 
locations; and 

a non-linear model of the sensitivity of a human visual system's ability to perceive 
image detail at eccentric visual angles. 

The above combination of elements provides for a high quality yet low bandwidth 
of a video signal comprising a facial image. 

18. Claim 5, 8, 9, 12, 24, 25, 30, 32-36, 38, 43, 46-51, 55-57 and 61 are objected to 
as being dependent upon a rejected base claim, but would be allowable if rewritten in 
independent form including all of the limitations of the base claim and any intervening 
claims. 



19. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. See In re Goodman, 1 1 
F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970);and, In re Thorington, 
418 F.2d 528, 163 USPQ 644 (CCPA 1969). 



Double Patenting 
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A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) may be 
used to overcome an actual or provisional rejection based on a nonstatutory double 
patenting ground provided the conflicting application or patent is shown to be commonly 
owned with this application. See 37 CFR 1 .130(b). 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

20. All of the pending claims are rejected under the judicially created doctrine of 
obviousness-type double patenting as being unpatentable over the patented claims U.S. 
Patent No. 6,1 73,069 B1 . Because of the number and variation between of claims, one 
claim will be exemplified. For example, claim 37 is rejected under the judicially created 
doctrine of obviousness-type double patenting as being unpatentable over claim 18 of 
U.S. Patent No. 6,173,069 B1 . Although the conflicting claims are not identical, they are 
not patentably distinct from each other because claim 18 teaches all of the elements of 
claim 37. The filing of a proper and timely terminal disclaimer will serve to overcome 
this rejection immediately. 

Conclusion 

21 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian P. Werner whose telephone number is 703-306- 
3037. The examiner can normally be reached on M-F, 8:00 - 4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H. Boudreau can be reached on 703-305-4706. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Brian Werner 
Primary Examiner 
Art Unit 2621 
March 1 1 , 2004 
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